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8.1.0 Introduction 

In real life, we work with data that are affected by randomness, and we need to extract 
information and draw conclusions from the data. The randomness might come from a variety 
of sources. Here are two examples of such situations: 

1. Suppose that we would like to predict the outcome of an election. Since we cannot poll 
the entire population, we will choose a random sample from the population and ask 
them who they plan to vote for. In this experiment, the randomness comes from the 
sampling. Note also that if our poll is conducted one month before the election, another 
source of randomness is that people might change their opinions during the one month 
period. 

2. In a wireless communication system, a message is transmitted from a transmitter to a 
receiver. However, the receiver receives a corrupted version (a noisy version) of the 
transmitted signal. The receiver needs to extract the original message from the received 
noisy version. Here, the randomness comes from the noise. 

Examples like these are abundant. Dealing with such situations is the subject of the field of 
statistical inference. 

Statistical inference is a collection of methods that deal with drawing conclusions from data 
that are prone to random variation. 

Clearly, we use our knowledge of probability theory when we work on statistical inference 
problems. However, the big addition here is that we need to work with real data. The 
probability problems that we have seen in this book so far were clearly defined and the 
probability models were given to us. For example, you might have seen a problem like this: 

Let X be a normal random variable with mean /I — 100 and variance <7 2 = 15 . 

Find the probability that X > 110 . 

In real life, we might not know the distribution of X, so we need to collect data, and from the 
data we should conclude whether X has a normal distribution or not. Now, suppose that we 
can use the central limit theorem to argue that X is normally distributed. Even in that case, we 
need to collect data to be able estimate fi and <7. 

Here is a general setup for a statistical inference problem: There is an unknown quantity that 
we would like to estimate. We get some data. From the data, we estimate the desired quantity. 
There are two major approaches to this problem: 


1. Frequentist (classical) Inference: In this approach, the unknown quantity 9 is assumed 
to be a fixed quantity. That is, 9 is a detenninistic (non-random) quantity that is to be 
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estimated by the observed data. For example, in the polling problem stated above we 
might consider 9 as the percentage of people who will vote for a certain candidate, call 
him/her Candidate A. After asking 71 randomly chosen voters, we might estimate 6 by 



n 




where Y is the number of people (among the randomly chosen voters) who say they 
will vote for Candidate A. Although 9 is assumed to be a non-random quantity, our 

A 

estimate of 9, which we show by 0 is a random variable, because it depends on our 
random sample. 


2. Bayesian Inference: In the Bayesian approach the u nkn own quantity 0 is assumed to be 
a random variable, and we assume that we have some initial guess about the distribution 
of 0 . After observing the data, we update the distribution of 0 using Bayes' Rule. 

As an example, consider the communication system in which the information is 
transmitted in the fonn of bits, i.e., 0's and l's. Let's assume that, in each transmission, 
the transmitter sends a 1 with probability p, or it sends a 0 with probability 1 — p. 
Thus, if 0 is the transmitted bit, then 0 ~ BerTlOulli (p) . At the receiver, X, 
which is a noisy version of 0 , is received. The receiver has to recover 0 from X. 

Here, to estimate 0 , we use our prior knowledge that 0 ~ Bernoulli (p). 


In summary, you may say that frequentist (classical) inference deals with estimating non- 
random quantities, while Bayesian inference deals with estimating random variables. We will 
discuss frequentist and Bayesian approaches more in detail in this and the next chapter. 
Nevertheless, it is important to note that both approaches are very useful and widely used in 
practice. In this chapter, we will focus on frequentist methods, while in the next chapter we 
will discuss Bayesian methods. 

<— previous 
next —> 


https://www.probabilitycourse.com/chapter8/8_1_0Jntro.php 


2/2 



